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INTRODUCTION 


Computer-generated displays are becoming increasingly popular in aerospace applications. 
The use of stereo 3-D technology provides an opportunity to present depth perceptions which oth- 
erwise might be lacking. In addition, the third dimension could also be used as an additional 
dimension along which information can be encoded. 

Historically, the stereo 3-D displays have been used in entertainment, in experimental facil- 
ities, and in the handling of hazardous waste. In the last example, the source of the stereo images 
generally has been remotely controlled television camera pairs. 

This paper describes the development of a stereo 3-D pictorial primary flight display used 
in a flight-simulation environment. The purpose of this research is to investigate the applicability 
of stereo 3-D displays for aerospace crew stations to meet the anticipated needs of the 2000-2020 
time frame. Although the actual equipment that could be used in an aerospace vehicle is not cur- 
rently available, the laboratory research is necessary to determine where stereo 3-D enhances the 
display of information and how the displays should be formatted. 


HARDWARE/SOFTWARE CONFIGURATION 


The hardware consists of a VAX 1 1/780 computer, an Adage 3000 raster programmable 
display generator (PDG), and a Stereographies 3-D display stereoscopic system. A FORTRAN 
aircraft simulation is used to provide parameters to the display programs residing in the 
Adage 3000. The display programs are written in a "C" language known as ICROSS-3000, with 
a graphics-enhancement package known as the Real-Time Animation Package (RAP). (RAP is a 
proprietary software product developed at the Research Triangle Institute.) 

The Stereographies display uses liquid crystal shuttered glasses and specially adapted 
hardware which divides each video frame into two fields corresponding to the left- and right-eye 
views, each at half the resolution. The PDG outputs a 60-Hz repeat field, 512 x 512 pixel image. 
The stereo display system converts this input to a 120-Hz repeat field, 216 x 512 pixel output with 
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alternating left- and right-eye fields. Figures 1 and 2 show a monocular version of the display. 
Figure 3 shows a similar display with left- and right-eye stereo views as they would appear on a 
conventional 60-Hz monitor. The Stereographies system converts the input shown in figure 3 and 
generates the stereo pairs similar to those in figures 1 and 2, but with only half the vertical resolu- 
tion. The liquid crystal shuttered glasses are synchronized so that each eye sees only one of the 
stereo views. 

A stereo image pair contains twice the information contained in a monocular image. There- 
fore, on a system with limited video bandwidth, either the video frame rate or the number of lines 
must be reduced when stereo displays are being generated. The current system maintains frame 
rate by halving the number of lines. Flicker, which was a problem with other systems, is thus 
eliminated. The system also performs the conversion of the video signal, and the PDG responds 
as if it were outputting its customary 60-Hz repeat field image. The liquid crystal shutter tech- 
nology is much faster than the video frame-rate-display capabilities; therefore, the stereo system 
does not impose any bandwidth limitations. 


DISPLAY FEATURES 


The main features of the display are an own-ship symbol, a perspective follow-me target 
ship, two different 3-D tracks showing the path of the target ship, a ground grid around the 
runway, a pitch grid on both the left and right sides of the display, and digital readouts for 
altitude/heading/airspeed. The digital readouts display the instantaneous values for the own-ship 
and the desired preprogrammed flightpath. Because the own-ship remains fixed relative to the 
other display elements, an inside-out (i.e., moving horizon) display is represented. 


Generating the Stereo 3-D Effect 

The display program needs to generate the left- and right-eye views of the display. Given 
distinct x, y locations of each eye, the calculation of the viewing transformations are described by 
Foley and Van Dam (ref. 3). 

Two parameters are used to control the stereo 3-D effect: zero-parallax distance and inter- 
ocular separation. In general, parallax refers to the positional discrepancy in the left- and right-eye 
views of a point in the display. The parallax is zero when the corresponding points in each view 
occupy the same relative screen location. Points in the display at the zero-parallax distance from 
the eye appear to lie in the plane of the screen. Points closer to the pilot than the zero-parallax dis- 
tance appear to lie in front of the screen, while points farther from the pilot appear to lie beyond the 
screen. In addition, the interocular distance controls the apparent relative depth of objects in the 
display. The greater the interocular distance, the more powerful the stereopsis effect. By com- 
paring the apparent depth of the target ship with the own-ship symbol, the pilot has an indication of 
position error. This stereo 3-D effect reinforces the depth cue provided by the relative size of the 
perspective target ship. 

When viewing objects in the natural environment, the eyes must perform the separate func- 
tions of converging and focusing on a point of interest. In a stereoscopic display, although the 
eyes must converge on an object, they focus on the plane of the screen regardless of the apparent 
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distance. One requirement of a stereo 3-D display is to minimize that disparity (ref. 4). This is 
accomplished by keeping the principal objects near the zero-parallax distance where the focus and 
convergence relationship is correct. 

After setting the zero-parallax distance at the desired distance from the aircraft to the target 
ship, the size of the target ship becomes a distance cue. For example, if the pilot is following too 
closely, the target ship appears larger on the screen and projects out of the plane of the screen 
towards the pilot. Conversely, if the pilot drops behind the target ship, it appears to shrink in size 
and recede into the background. The combination of stereo and size cue serves as an important 
error indicator. 


In this display, the zero-parallax distance is set to a nominal following distance. The inter- 
ocular distance was established empirically at 8 ft. Moving die eyes that far apart is equivalent to 
shrinking the scale of the scene proportionally. Such distortions enhance the pilot s ability to per- 
ceive the sensations of depth. They are also necessary because of inherent limitations of the hard- 
ware. The precision in rendering the left- and right-eye views is limited both by the display reso- 
lution and the arithmetic precision of the display processor (i.e., 16-bit fixed point). 

If a fixed time lag rather than a fixed distance is desired, the zero-parallax also could be 
dynamic. In that case, the zero-parallax distance would be a function of the time lag and the 
instantaneous velocity of the target ship. 


Within the 3-D display, apparent depth had to be assigned to 2-D symbols such as digital 
readouts and the pitch scale. Two possible choices are the zero-parallax distance or the maximum 
distance. If they are set at the zero-parallax distance (i.e., drawn with the same left- and nght-eye 
view), they would be perceived by the pilot as if they were being looked past in order to see the 
part of the 3-D display beyond the zero-parallax distance. Earlier informal evaluation has shown 
the resulting perception to be disorienting. Instead, by placing the 2-D symbols at the maximum 
distance, they appear natural and unobtrusive. 

Care must be taken when defining the left- and right-eye transformations. Figure 4 illus- 
trates two ways of conceptualizing the transformations. In figure 4a, the views are converged by 
rotation of the viewing pyramid. In figure 4b the viewing pyramids are sheared. The latter 
approach is preferred, as the projection planes in each view remain parallel. Achieving conver- 
gence by rotation creates artifacts which can not only cause eye fatigue, but also can interfere with 
the pilot’s perception of depth (ref. 4). Although the rotation method is easier to implement, the 
shearing approach has become the standard in 3-D graphics software (refs. 1 and 2). 


Own-Ship Symbol 

Figure 5 shows the evolution of the own-ship symbol. The original configuration, fig- 
ure 5a, presented the pilot with two problems. First, it was impossible to perceptually fuse the 
right- and left-eye viewpoints to form the 3-D image. This fusion problem was surprising, 
because the signposts also were made of single, straight lines, but there was no problem with their 
visual fusion. An additional problem was that the own-ship symbol tended to get lost in the 
display. The signpost symbol was constructed of perpendicular horizontal and vertical lines; the 
same was true of the own-ship symbol. Therefore, there were many instances in which the own- 
ship symbol would overlay the signposts and could not be perceived. 


39-3 


In order to increase the pilot's ability to perceive the own-ship symbol, the center slanted 
lines were drawn as shown in figure 5b. Although the ability to perceive the symbol was greatly 
increased, there was still the problem of inability to visually fuse the stereo 3-D image. 

Figure 5c was originally constructed to further enhance the pilot's ability to perceive the 
own-ship symbol; it worked. A serendipitous benefit was that the symbol now visually fused. At 
this time there is no theoretical explanation for the fusion phenomena. 


INITIAL RESEARCH 


The initial research with the display will be a study of recovery from flightpath offset. 
Pilots will be initiated on the nominal flightpath. After 2 sec, they suddenly will experience a 
flightpath offset. They will be required to make the stick input to rejoin the nominal flightpath. 
Visually evoked potentials will be triggered from the sudden flightpath offset. In addition, reaction 
times, response accuracy, and a projected workload estimate also will be recorded. The Subjective 
Workload Assessment Technique (SWAT) will be used for the workload estimate (refs. 5 and 6). 
A test for stereoscopic acuity will be administered prior to data collection. Recent anecdotal 
evidence indicated that some subjects tend to lose the ability to use the stereoscopic cue after 
prolonged exposure to it. Therefore, stereoscopic acuity also will be measured immediately after a 
long series of trials with the stereo 3-D cues. 

In addition to using stereo 3-D or monocular cues as an independent variable, the inclusion 
or exclusion of the target ship will be the second independent variable. The last independent vari- 
able will be the pathway. There either will be the signpost or a monorail pathway for the subjects 
to follow. 


FUTURE RESEARCH 


The initial research will use the stereo 3-D cues to represent geographic information. In the 
"real world," objects are geographically separated by space, and the displays will attempt to create 
the perception of that geographic separation. 

In contrast, one line of future research will use the third dimension as a dimension to 
encode new information for the pilot. For example, presume that there is a pictorial display which 
is entirely in the plane of the screen and that depth perception is simulated with monocular cues 
such as linear perspective. If a pilot were using that display in a current aircraft, and if the airspeed 
were to get too low, an audio display (i.e., a hom) would sound. The audio display is an alerting 
display, and the pilot must know to then look at the visual display for speed. 

However, part of the pictorial display is a box with digital readouts for instantaneous actual 
and desired airspeed. Using the same airspeed error example, the box with the airspeed would 
modulate in the third dimension (i.e., along the z-axis) as the alerting cue instead of using the audio 
cue as the alerting cue. In this manner, new information would be presented to the subjects in the 
third dimension. 
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From a human factors perspective, that is a potential way of decreasing the total number of 
cockpit displays and also to make the alerting cues more nearly intuitively obvious. There are 
many research questions to be addressed. First, can it be demonstrated that the proposed use of 
stereo 3-D is quantifiably better than the use of audio alerting cues? Some of the other questions 
concern the rate and perceived depth of modulation in the third dimension. For example, should 
the rate or perceived depth of modulation be proportional to the amount of error? Should the mod- 
ulation only be from the plane of the screen towards the pilot or should it also modulate from the 
plane of the screen away from the pilot? 

Other uses of stereo 3-D also are possible. The natural use of stereo 3-D is to represent 
the 3-D geography. Part of the true test of the technology will be to go beyond that approach and 
determine if there are more effective applications. 
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Figure 1.- Monocular "monorail" display. 



Figure 2.- Monocular "signpost" display. 
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Figure 3.- Stereo display as seen on a conventional CRT. 




Figure 4.- Generation of stereo pairs by eye rotation (a); generation of pairs by shearing the view- 
ing pyramid (b). 
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Figure 5.- Evolution of own-ship symbol: Stereo pairs for (a) and (b) would not visually fuse; 

(c) would visually fuse. 
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